Performance evaluating apparatus, performance evaluating method, and program

ABSTRACT

A performance evaluating apparatus for a rule is provided, which is capable of evaluating a business value of a rule applied to the operation and management of an information system through autonomous control. The performance evaluating apparatus for a rule collects, respectively before activation processing that is prescribed by a rule and after completion of the execution of the processing, a rule execution history and a system state information indicating a state of a monitored information system that is necessary for calculation of the business value. The business value is calculated from the collected system state information and rule execution history by a given calculation formula.

FIELD OF THE INVENTION

The present invention relates to a performance evaluating apparatus foran information system composed of multiple computers and a method ofevaluating the performance of the information system.

BACKGROUND OF THE INVENTION

Rule-based autonomous operation is a technology for simplifying theoperation and management of an information system. Rule-based autonomouscontrol is based on a rule dictating that processing (an action),including a preset operation and management operation, is executed whena certain condition is met. A rule used in autonomous control technologyis evaluated for its effect based on information, such as a result ofapplying the rule, and is improved in accordance with the evaluation.This method is called feedback method and is attracting attention. Forinstance, Japanese Patent Laid-open Publication No. 2005-354280(hereinafter referred to as Patent Document 1) discloses a techniquerelated to improvement through feedback of rule application historyinformation, which contains information about a period during which arule called a policy rule is applied, and information indicating theresult of applying the rule. According to the technique, the applicationhistory information is collected and analyzed in order to collectstatistical information about a condition associated with the rule, andthe statistical information is used to change condition and actionconstitute the rule.

SUMMARY OF THE INVENTION

Rule evaluation often employs, for example, a time point at which a ruleis activated, an activation count, accumulated operating time, or anachievement rate for an objective set in advance in an informationsystem to be controlled. The rule that makes the information systemconsume a large amount of computer resources and accordingly raises thecost, or that causes large power consumption by being in operation andthus puts a heavy burden on the natural environment, is not always givena high score. Therefore there is a need to evaluate a rule from moreviewpoints and more pieces of information relating to an informationsystem to be controlled, than only the rule activation time, activationcount, accumulated operating time, or objective achievement rate.

For example, collecting and saving every piece of state informationabout an information system to be controlled makes it possible toevaluate a rule based on the saved information from various viewpointsincluding business viewpoint such as the rule's contribution to salesfigures of a service that the information system provides, contributionto the running cost of the information system, and contribution to powerconsumption during running of the information system. However, there isa wide variety of information that indicates the state of an informationsystem, and information that changes at short time intervals has to becollected at short intervals. The amount of data to be collected istherefore huge and it is impossible in practice to collect every pieceof state information about an information system to be controlled.

The present invention has been made in view of the above-mentionedcircumstances, and therefore an object of the present invention is toevaluate a business value of a rule for use in operation managementthrough autonomous control based on information that is efficientlycollected for evaluation from many viewpoints including a businessviewpoint, and to numerically show business value from those viewpoints.

The present invention attains the above-mentioned object as follows:

Before processing, which is prescribed by a rule, is activated, systemstate information, which indicates the state of a system to be monitoredand is necessary for business value evaluation, is collected and storedin association with information that identifies the processing to beactivated, as well as information that indicates that associatedinformation is system state information collected upon activation of theprocessing. After the processing prescribed by the rule is finished,system state information indicating the state of the monitored system iscollected and stored in association with information that identifies thefinished processing, as well as information that indicates thatassociated information is system state information collected uponcompletion of the execution of the processing. The business value of therule is calculated from these two pieces of system state information.

For example, according to a first aspect of the present invention, thereis provided a performance evaluating apparatus for a rule used in anautonomous control system that controls a system that is to be monitoredby executing processing that is prescribed in a preset rule inaccordance with the state of the monitored system, which is composed ofmultiple computer resources, the performance evaluating apparatusincluding: a rule executing unit which executes processing prescribed inan associated rule in accordance with the state of the monitored systemand which stores a history about the executed processing as a ruleexecution history; a rule execution history obtaining unit which obtainsa rule execution history from the rule executing unit to store theobtained history in a rule operation history storing unit; a systemstate information collecting unit which collects system stateinformation indicating the state of the monitored system to send thecollected system state information to the rule operation history storingunit; and an effect calculating unit which calculates a business valuefrom the stored system state information and execution history by acalculation formula, in which: the rule executing unit has a ruleprocessing unit, an execution history storing unit, and an executionhistory transmitting unit, the rule processing unit sending, inexecuting processing prescribed in a rule, to the system stateinformation collecting unit, an activation notification containing aprocessing ID with which the processing to be executed is identified,and then starting execution of processing identified by the processingID, and after finishing execution of the processing prescribed in therule, sending an execution completion notification containing aprocessing ID of the finished processing to the system state informationcollecting unit, the execution history storing unit storing a ruleexecution history in which information about processing executed by therule processing unit is associated with a processing ID of thisprocessing, the execution history transmitting unit sending, uponrequest from the rule execution history obtaining unit, a rule executionhistory in the execution history storing unit to the rule executionhistory obtaining unit; upon reception of the activation notificationfrom the rule executing unit, the system state information collectingunit collects system state information and sends the collected systemstate information to the rule execution history obtaining unit inassociation with a processing ID that is contained in the receivedactivation notification and information that indicates that associatedinformation is system state information collected upon activation ofprocessing identified by this processing ID; upon reception of theexecution completion notification from the rule executing unit, thesystem state information collecting unit collects system stateinformation and sends the collected system state information to the ruleexecution history obtaining unit in association with a processing IDthat is contained in the received execution completion notification andinformation that indicates that associated information is system stateinformation collected upon completion of the execution of processingidentified by this processing ID; and the effect calculating unitcalculates a business value from the stored system state information andexecution history by a calculation formula.

For example, according to a second aspect of the present invention,there is provided a performance evaluating method for a performanceevaluating apparatus for a rule used in an autonomous control systemthat controls a system to be monitored by executing processing that isprescribed in a preset rule in accordance with the state of themonitored system which is composed of multiple computer resources, whichincludes: a rule executing unit which executes processing prescribed inan associated rule in accordance with the state of the monitored systemand which stores a history about the executed processing as a ruleexecution history; a rule execution history obtaining unit which obtainsa rule execution history from the rule executing unit to store theobtained history in a rule operation history storing unit; a systemstate information collecting unit which collects system stateinformation indicating the state of the monitored system to send thecollected system state information to the rule operation history storingunit; and an effect calculating unit which calculates a business valuefrom the stored system state information and execution history by acalculation formula, the method including the steps of: causing the ruleexecuting unit to execute a rule processing step, an execution historystoring step, and an execution history transmitting step, the ruleprocessing step including sending, in executing processing prescribed ina rule, to the system state information collecting unit, an activationnotification containing a processing ID with which the processing to beexecuted is identified, and then, starting execution of processingidentified by the processing ID, and after finishing execution of theprocessing prescribed in the rule, sending an execution completionnotification containing a processing ID of the finished processing tothe system state information collecting unit, the execution historystoring step including storing in an execution history storing unit arule execution history in which information about processing executed inthe rule processing step is associated with a processing ID of thisprocessing, the execution history transmitting step including sending,upon request from the rule execution history obtaining unit, a ruleexecution history in the execution history storing unit to the ruleexecution history obtaining unit; causing the system state informationcollecting unit to collect, upon reception of the activationnotification from the rule executing unit, system state information andsend the collected system state information to the rule executionhistory obtaining unit in association with a processing ID that iscontained in the received activation notification and information thatindicates that associated information is system state informationcollected upon activation of processing identified by this processingID; and collect upon reception of the execution completion notificationfrom the rule executing unit, system state information and send thecollected system state information to the rule execution historyobtaining unit in association with a processing ID that is contained inthe received execution completion notification and information thatindicates that associated information is system state informationcollected upon completion of the execution of processing identified bythis processing ID; and causing the effect calculating unit to calculatea business value from the stored system state information and executionhistory by a calculation formula.

For example, according to a third aspect of the present invention, thereis provided a program that controls a performance evaluating apparatusfor a rule used in an autonomous control system that controls a systemto be monitored by executing processing that is prescribed in a presetrule in accordance with a state of the monitored system which iscomposed of multiple computer resources, the program causing theperformance evaluating apparatus for a rule to function as: a ruleexecuting function which executes processing prescribed in an associatedrule in accordance with the state of the monitored system and whichstores a history about the executed processing as a rule executionhistory; a rule execution history obtaining function which obtains arule execution history from the rule executing function to store theobtained history in a rule operation history storing unit; a systemstate information collecting function which collects system stateinformation indicating the state of the monitored system to send thecollected system state information to the rule operation history storingunit; and an effect calculating function which calculates a businessvalue from the stored system state information and execution history bya calculation formula, and in the program: the rule executing functionhas a rule processing function, an execution history storing function,and an execution history transmitting function, the rule processingfunction sending, in executing processing prescribed in a rule, to thesystem state information collecting function, an activation notificationcontaining a processing ID with which the processing to be executed isidentified, and then, starting execution of processing identified by theprocessing ID, and after finishing execution of the processingprescribed in the rule, sending an execution completion notificationcontaining a processing ID of the finished processing to the systemstate information collecting function, the execution history storingfunction storing in the execution history storing unit a rule executionhistory in which information about processing executed by the ruleprocessing function is associated with a processing ID of thisprocessing, the execution history transmitting function sending, uponrequest from the rule execution history obtaining function, a ruleexecution history in the execution history storing unit to the ruleexecution history obtaining function; upon reception of the activationnotification from the rule executing function, the system stateinformation collecting function collects system state information andstores in the rule operation history storing unit the collected systemstate information in association with a processing ID that is containedin the received activation notification and information that indicatesthat associated information is system state information collected uponactivation of processing identified by this processing ID; uponreception of the execution completion notification from the ruleexecuting function, the system state information collecting functioncollects system state information and stores in the rule operationhistory storing unit the collected system state information inassociation with a processing ID that is contained in the receivedexecution completion notification and information that indicates thatassociated information is system state information collected uponcompletion of the execution of processing identified by this processingID; and the effect calculating function calculates a business value fromthe stored system state information and execution history by acalculation formula.

According to a performance evaluating apparatus of the presentinvention, a rule applied to the operation and management of aninformation system can be evaluated from a business viewpoint.

BRIEF DESCRIPTION OF THE DRAWINGS

In the accompanying drawings:

FIG. 1 is a system configuration diagram showing an example of aconfiguration of an information system and a performance evaluatingapparatus according to an embodiment of the present invention;

FIG. 2 is a conceptual diagram illustrating a business operation systemof a Web three-tier model;

FIG. 3 is a diagram showing an example of a configuration of data thatis stored in a rule storing unit;

FIG. 4 is a flow chart showing an example of an operation of aperformance evaluating apparatus for a rule;

FIG. 5 is a flow chart showing an example of the operation of theperformance evaluating apparatus for a rule;

FIG. 6 is a diagram illustrating an example of a configuration of datathat is obtained by a system state information collecting unit in StepS302;

FIG. 7 is a diagram illustrating an example of a configuration of datathat is collected by the system state information collecting unit inStep S304;

FIG. 8 is a diagram illustrating an example of a configuration of datathat is collected by the system state information collecting unit inStep S305;

FIG. 9 is a diagram illustrating an example of a data configuration ofresource configuration/state information which is stored in a ruleoperation history storing unit;

FIG. 10 is a diagram illustrating an example of a data configuration ofinformation that indicates a service utilization state and is stored inthe rule operation history storing unit;

FIG. 11 is a diagram illustrating an example of a data configuration ofan execution history which is stored in the rule operation historystoring unit;

FIG. 12 is a flow chart showing an example of data processing that isexecuted by an effect calculating unit;

FIG. 13 is a diagram showing an example of a calculation formula that isused in the data processing;

FIG. 14 is a diagram illustrating an example of a configuration of datasuch as financial cost and power consumption that are defined for eachresource;

FIG. 15 is a diagram illustrating an example of a configuration of datafor calculating a business value;

FIG. 16 is a diagram showing an example of a configuration ofinformation that is output from the performance evaluating apparatus fora rule;

FIG. 17 is a diagram showing an example of a hardware configuration forrealizing functions of the performance evaluating apparatus for a rule;and

FIG. 18 is a flow chart showing another example of execution historycollection processing which is executed by the performance evaluatingapparatus for a rule.

DESCRIPTION OF THE PREFERRED EMBODIMENT

An embodiment of the present invention will be described below.

FIG. 1 is a system configuration diagram showing an example of aconfiguration of an information system 1 and a performance evaluatingapparatus 2 according to an embodiment of the present invention. Asshown in the drawing, the information system 1 of this embodiment hasmultiple load balancers 13 and multiple computer resources 16, whichconstitute an information system, a router 12, which connects theinformation system to the Internet 11, and user terminals 10, whichaccess the information system via the Internet 11.

The router 12, the load balancers 13, and the computer resources 16 areconnected to one another via a business operation network 14 and via amanagement network 15. The performance evaluating apparatus for a rule 2is connected to the management network 15.

The management network 15 is a network used by the performanceevaluating apparatus for a rule 2 to manage each load balancer 13 andeach computer resource 16. The business operation network 14 is anetwork used by each computer resource 16 to provide a service to theuser terminals 10. The business operation network 14 and the managementnetwork 15 may be built separately using physically different cables ormay be built on the same cable as logically different networks.

The router 12 has a routing storing unit 120 which stores for each loadbalancer 13 the association relation between a global IP address and alocal IP address. The router 12 sends a request received from one of theuser terminals 10 via the Internet 11 to the load balancer 13 whoselocal IP address is, according to the association relation stored in therouting storing unit 120, associated with a global IP address that isspecified as the destination of this request.

The router 12 also receives from the load balancer 13 a result ofprocessing the request, and sends the result to the user terminal 10that is the sender of this request via the Internet 11. Other functionsof the router 12 are the same as those of any existing router, and willnot be described in detail. Each load balancer 13 forms a businessoperation system with multiple computer resources 16 via the businessoperation network 14.

A business operation system in this embodiment is a business operationsystem of a Web three-tier model as shown in FIG. 2. The businessoperation system of the Web three-tier model is composed of Web tierlogical components 30A, which perform processing of accepting requestsfrom users through Web browsers, application (AP) tier logicalcomponents 30B, which process services provided to users, and a database(DB) tier logical component 30C.

The Web tier logical components 30A are Web servers that are scalable(scale in/scale out, meaning that their number can be increased orreduced). The application tier logical components 30B are scalable(scale in/scale out) application servers. The database tier logicalcomponent 30C is a database server.

The computer resources 16, in which Web containers are installed, areallotted to the Web servers. The computer resources 16, in whichapplication containers are installed, are allotted to the applicationservers. The computer resource 16, in which a database management systemis installed, are allotted to the database servers. The Web servers andthe AP servers may be allocated physically to the same servers. Theconfiguration of the business operation system is not limited to the Webthree-tier model and may be any configuration.

Each load balancer 13 has a distribution destination storing unit 130 toregister information about which computer resource 16 it can beconnected to, and a measuring/monitoring unit 131 which measures andmonitors metrics of the load balancer 13.

Each load balancer 13 sends, via the business operation network 14, arequest received from the router 12 to one of the computer resources 16that are registered in the distribution destination storing unit 130, tohave this computer resource 16 process the request. Also, each loadbalancer 13 receives, via the business operation network 14, from one ofthe computer resources 16 registered in the distribution destinationstoring unit 130, a result of processing a request, and sends, via therouter 12, the received processing result to the sender of the request.

Each load balancer 13 executes processing such as updating thedistribution destination storing unit 130 in accordance with anoperation received from the performance evaluating apparatus for a rule2 via the management network 15. The measuring/monitoring unit 131measures and monitors information of its own load balancer 13, such asperformance information including the operating state and the CPUutilization rate, and connection information including the count ofrequests received from the router 12, and when a condition defined inadvance is met, the measuring/monitoring unit 131 notifies theperformance evaluating apparatus for a rule 2 via the management network15 of the fact as an event.

In this embodiment, the measuring/monitoring unit 131 monitors, asevents, whether or not there is a change in an operating state(activation or shutdown) of its own load balancer 13, whether or not theload level has exceeded a given reference value, and the like. Otherfunctions of the load balancers 13 are the same as those of any loadbalancer used in existing load-balancing systems, and will not bedescribed in detail.

Each computer resource 16 uses an installed program to process a requestreceived via the business operation network 14, and sends a result ofprocessing the request to the load balancer 13 or other computerresource 16 that forms, together with this computer resource 16, abusiness operation system. Also, each computer resource 16 controlsexecution of a program in accordance with an operation received from theperformance evaluating apparatus for a rule 2 via the management network15.

As shown in FIG. 1, each computer resource 16 has a network interface(IF) unit 160 to connect with the business operation network 14 and withthe management network 15, a measuring/monitoring unit 161, and aprogram executing unit 162.

The measuring/monitoring unit 161 measures and monitors information ofits own computer resource 16, such as performance information includingthe operating state and the CPU utilization rate, power consumption, andconnection information including the count of requests received from therouter 12, and when a condition defined in advance is met, themeasuring/monitoring unit 161 notifies the performance evaluatingapparatus for a rule 2 via the management network 15 of the fact, as anevent. In this embodiment, the measuring/monitoring unit 161 monitors asevents whether or not there is a change in an operating state(activation or shutdown) of its own computer resource 16, whether or notthe load level has exceeded a given reference value, and the like.

The program executing unit 162 uses a program installed in its computerresource 16 to process a request received via the business operationnetwork 14, and sends, via the business operation network 14, a resultof processing the request to the load balancer 13 or other computerresource 16 that forms a business operation system together with its owncomputer resource 16.

Programs installed in the computer resources 16 in this embodiment are aWeb container, an application container, and a database managementsystem. As mentioned above, the computer resource 16 in which a Webcontainer is installed is allocated as a Web server, the computerresource 16 in which an application container is installed is allocatedas an application server, and the computer resource 16 in which adatabase management system is installed is allocated as a databaseserver. Other functions of the computer resources 16 are the same asthose of any computer resource used in existing load-sharing systems,and will not be described in detail.

The performance evaluating apparatus 2 evaluates the performance for arule applied to the operation and management of the information system1. The performance evaluating apparatus for a rule 2 has an input/outputunit 20, a configuration information management unit 21, a configurationinformation storing unit 22, an event receiving unit 23, a ruleexecuting unit 24, a rule execution history obtaining unit 25, aspecified information receiving unit 251, a rule operation historystoring unit 26, a system state information collecting unit 27, aneffect calculating unit 28, a calculation formula input receiving unit280, and a network interface (IF) unit 29.

The input/output unit 20 handles inputs to and outputs from theperformance evaluating apparatus for a rule 2 by, for example, acceptingan operation input from a user who is operating the performanceevaluating apparatus for a rule 2, or outputting information that iscreated by the performance evaluating apparatus for a rule 2. Thenetwork IF unit 29 has a function of enabling the performance evaluatingapparatus for a rule 2 to communicate over the management network 15with an information system to be monitored (a business operation systemconstituted of multiple load balancers 13 and multiple computerresources 16).

The configuration information storing unit 22 stores the systemconfiguration information of an information system to be monitored and abusiness operation system hosted by this information system. The systemconfiguration information stored in the configuration informationstoring unit 22 contains resource configuration information, componentconfiguration information, and component allocation information. Theresource configuration information includes information about ITresources constituting the information system (including informationthat indicates the operating state of the resources) and inter-resourceconnection information. The component configuration information concernslogical components in an application program forming the businessoperation system. The component allocation information concerns whichlogical component is allocated to which physical computer resource 16.

The configuration information management unit 21 monitors configurationinformation of an information system and, when the configurationinformation changes, changes information stored in the configurationinformation storing unit 22. The event receiving unit 23 receives eventsfrom the load balancers 13 or the computer resources 16, and suppliesthe events to the rule executing unit 24.

Events received by the event receiving unit 23 in this embodimentinclude changes detected by the measuring/monitoring units of the loadbalancers 13 or the computer resources 16, for example, changes in loadsuch as a change in CPU utilization rate, an increase or decrease inused memory amount, an increase or decrease in count of page faultsoccurring upon memory access, and an increase or decrease in I/O waitingtime, and changes in operating state of IT resources including programsthat provide business operation services.

The rule executing unit 24 has a rule storing unit 240, a ruleprocessing unit 241, an execution history storing unit 242, and anexecution history transmitting unit 243.

The rule storing unit 240 stores, for example, as shown in FIG. 3, arule used in the operation and management of an information system inassociation with a RULE_ID 2400 that identifies each rule. Each rulecontains a TRIGGER_CONDITION 2401, which indicates a condition ofapplying processing, an ACTION_ID 2403, which indicates processing (anaction) that is executed when the information system fulfills thiscondition, and APPLY_TO 2402, which indicates a resource that executesthis processing.

In the example of FIG. 3, to a rule that has “A” as the RULE_ID 2400, anapplication condition “the CPU utilization rate of WebServer#1 is 20% orlower” is set as the TRIGGER_CONDITION 2401, “RESOURCE: WebServer#1” isspecified as the APPLY_TO 2402, and “scale-in the Web tier” is specifiedas the ACTION_ID 2403. In this embodiment, the character stringpreceding “:” in the APPLY_TO 2402, that is, “RESOURCE”, indicates thatthe type of IT resource to which the processing is applied is an ITresource such as a server.

Other types of IT resource to which processing is applied in thisembodiment include “SERVICE”, which is an identifier indicating aservice that is provided by a business operation system, “GROUP”, whichis an identifier indicating a group of multiple IT resources, and“APPLICATION_SYSTEM”, which is an identifier indicating the entirebusiness operation system. In this embodiment, processing of “scaling-inthe Webtier” means processing of reducing the count of Web servers thatconstitute the Web tier, to thereby free redundant IT resources.

Rules that have “B”, “C”, “D”, and “E” as the RULE_ID 2400 will bedescribed in a similar manner. To a rule whose RULE_ID 2400 is “B”, anapplication condition “the count of requests for a catalog service is100 or more per second” is set when a “catalog service”, which is aservice for displaying a catalog, is included in services provided by abusiness operation system. “SERVICE: Catalog” and “scale-out the Webtier” are specified as the APPLY_TO 2402 and as the ACTION_ID 2403,respectively, for the rule “B”.

Processing of “scaling-out the Web tier” in this embodiment meansprocessing of increasing Web servers in number to thereby lessen theload per Web server and enable the business operation system to processrequests from many more users.

To a rule whose RULE_ID 2400 is “C”, an application condition “the loadon the Web tier fluctuates wildly” is set, “GROUP: WebTier” indicating agroup of Web servers constituting the Web tier is specified as theAPPLY_TO 2402, and processing “adjust the load-balancing coefficient ofthe Web tier” is specified as the ACTION_ID 2403.

Processing of “adjusting the load-balancing coefficient of the Web tier”in this embodiment means processing of adjusting a load-balancingcoefficient, which is a settings parameter set to the load balancers 13for load balancing in each Web server, so that the load is evened outamong the multiple Web servers.

To a rule whose RULE_ID 2400 is “D”, an application condition “serverfailure” is set, “GROUP: Server” indicating a group of IT resourceswhose resource type is server machine is specified as the APPLY_TO 2402,and processing “switch to an auxiliary server” is specified as theACTION_ID 2403.

Processing of “switching to an auxiliary server” in this embodimentmeans processing of switching from a failed server to an auxiliaryserver which is on standby for the purpose of continuing the service.

To a rule whose RULE_ID 2400 is “E”, an application condition“occurrence of accident such as earthquake, tsunami, and fire isdetected” is set, “APPLICATION_SYSTEM: Onlineshop” indicating an entirebusiness operation system is specified as the APPLY_TO 2402, andprocessing “switching systems from primary to secondary (failover)” isspecified as the ACTION_ID 2403.

Processing of “switching systems from primary to secondary (failover)”in this embodiment means processing of switching from a primary sitewhere a business operation system is run, to a secondary site which isset up in advance as a backup site when the primary site is struck by adisaster, in order to have the secondary site take over the businessoperation.

The rule processing unit 241 receives an event from the event receivingunit 23, and searches rules in the rule storing unit 240 for anapplication condition that matches the received event. The ruleprocessing unit 241 then extracts processing that is stored in the rulestoring unit 240 in association with the retrieved applicationcondition.

The rule processing unit 241 next creates an instance as an executor ofthe extracted processing, and sends an activation notificationcontaining identification information of the created instance to thesystem state information collecting unit 27. After that, the ruleprocessing unit 241 makes the created instance execute the extractedprocessing. As the instance finishes the processing, the rule processingunit 241 sends an execution completion notification containingidentification information of the instance to the rule execution historyobtaining unit 25 and the system state information collecting unit 27.

When processing is executed in response to an event notified by theevent receiving unit 23, the rule storing unit 240 stores, in theexecution history storing unit 242, history information about theexecuted processing as an execution history. An execution historycontains identification information of an instance that has executed theprocessing in question, start time of the processing, end time of theprocessing, information indicating the executed processing, a result ofexecuting the processing, and the like.

Identification information of an instance is constituted ofidentification information of a rule containing an application conditionthat triggers the creation of this instance, and a sequence number whichis assigned to each newly created instance such that no two instanceshave the same sequence number. Accordingly, from instance identificationinformation, an instance can uniquely be identified and a rule thatcauses the creation of the instance can be identified as well.

The execution history transmitting unit 243 receives an executionhistory obtaining request containing instance identification informationfrom the rule execution history obtaining unit 25, and refers to theoperating state of the rule processing unit 241. When it is found as aresult that the rule processing unit 241 is not executing processing,the execution history transmitting unit 243 extracts, from the executionhistory storing unit 242, an execution history that is associated withthe instance identification information contained in the receivedexecution history obtaining request, and sends the extracted executionhistory to the rule execution history obtaining unit 25. On the otherhand, when the rule processing unit 241 is found to be executingprocessing, the execution history transmitting unit 243 sends a messageindicating that the rule processing unit 241 is performing processing,to the rule execution history obtaining unit 25 in response to therequest.

The system state information collecting unit 27 refers to configurationinformation stored in the configuration information storing unit 22 whenreceiving an activation notification from the rule processing unit 241.The system state information collecting unit 27 uses the configurationinformation to identify which resource is in operation, and obtains aresource utilization state and service utilization information from eachresource that is in operation via the management network 15.

The system state information collecting unit 27 stores, in the ruleoperation history storing unit 26, the obtained resource utilizationstate and service utilization information in association with theinstance identification information contained in the activationnotification and information indicating that associated information isthe state of the system before this instance starts execution ofprocessing.

When receiving an execution completion notification from the ruleprocessing unit 241, the system state information collecting unit 27refers to configuration information stored in the configurationinformation storing unit 22. The system state information collectingunit 27 uses the configuration information to identify which resource isin operation, and obtains a resource utilization state and serviceutilization information from each resource that is in operation via themanagement network 15.

The system state information collecting unit 27 stores, in the ruleoperation history storing unit 26, the obtained resource utilizationstate and service utilization information in association with theinstance identification information contained in the executioncompletion notification and information indicating that associatedinformation is the state of the system after this instance completes theexecution of processing.

The rule processing unit 241 and the system state information collectingunit 27, which are packaged as different processes in the performanceevaluating apparatus for a rule 2, are processed in parallel in theperformance evaluating apparatus for a rule 2. The rule processing unit241 and the system state information collecting unit 27 can thereforeexecute their respective processing without affecting each other'sprocessing.

The rule operation history storing unit 26 stores system stateinformation collected by the system state information collecting unit 27and an execution history obtained by the rule execution historyobtaining unit 25.

The rule execution history obtaining unit 25 has an execution historyacquisition executing unit 250 and a specified information receivingunit 251. The specified information receiving unit 251 receives, via theinput/output unit 20, from a user of the performance evaluatingapparatus for a rule 2, an execution history obtaining interval and anexecution count indicating how many times processing contained in a ruleshould be executed before the execution history is obtained. Thespecified information receiving unit 251 sends the received obtaininginterval and execution count to the execution history acquisitionexecuting unit 250.

The execution history acquisition executing unit 250 increments theexecution count of processing each time an execution completionnotification of the processing is received from the rule processing unit241. When the accumulated execution count reaches an execution countnotified by the specified information receiving unit 251 or higher, theexecution history acquisition executing unit 250 refers to the ruleoperation history storing unit 26 to identify instance identificationinformation for which a resource utilization state and serviceutilization information have been stored but no execution history hasbeen stored.

The execution history acquisition executing unit 250 sends an executionhistory obtaining request that contains the identified instanceidentification information to the execution history transmitting unit243. Receiving the requested execution history from the executionhistory transmitting unit 243, the execution history acquisitionexecuting unit 250 saves the time of reception and resets theaccumulated execution count.

In cases where a time equal to or longer than an obtaining intervalnotified by the specified information receiving unit 251 has elapsedsince the previous time an execution history was obtained, the executionhistory acquisition executing unit 250 refers to the rule operationhistory storing unit 26 to identify instance identification informationfor which a resource utilization state and service utilizationinformation have been stored but no execution history has been stored.

The execution history acquisition executing unit 250 sends an executionhistory obtaining request that contains the identified instanceidentification information to the execution history transmitting unit243. In cases in which the requested execution history is received fromthe execution history transmitting unit 243, the execution historyacquisition executing unit 250 saves the time of reception and resetsthe accumulated execution count.

The effect calculating unit 28 receives a rule operation history displayrequest from a user through the input/output unit 20, retrievesnecessary operation history information from the rule operation historystoring unit 26, formats the retrieved data into a necessary format withthe use of a preset calculation formula, and outputs the data throughthe input/output unit 20. The effect calculating unit 28 may have thecalculation formula input receiving unit 280 to receive, via theinput/output unit 20, input of a calculation formula that is defined bya user, and employ the entered calculation formula.

The operation of the performance evaluating apparatus for a rule 2 willbe described next.

FIGS. 4 and 5 are flow charts showing an example of the operation of theperformance evaluating apparatus for a rule 2. The premise of the flowcharts is that the configuration information storing unit 22 alreadystores configuration information of a business operation system createdby a user and configuration information of an IT resource platform of aninformation system that hosts the business operation system while therule definition information shown in FIG. 3 is already stored in therule storing unit 240 by the user.

The performance evaluating apparatus for a rule 2 starts the operationshown in the flow chart of FIG. 4 in response to the reception of anevent from one of the load balancers 13 or one of the computer resources16, by the event receiving unit 23. An event sent from the load balancer13 or the computer resource 16 in this embodiment matches an applicationcondition contained in one of rules that are stored in the rule storingunit 240. In the example of this flow chart, the rule processing unit241 obtains via the event receiving unit 23 an event notifying that theCPU utilization rate has dropped to 20% or lower in WebServer#1, whichis one of the Web servers that hosts the Web tier logical component 30A.

First, the event receiving unit 23 notifies the rule processing unit 241of the received event (S100). The rule processing unit 241 executes ruleactivation processing (S101). In Step S101, the rule processing unit 241refers to the rule storing unit 240 to identify a rule that contains anapplication condition matching the received event, creates an instancethat executes processing contained in the identified rule, and sends anactivation notification containing identification information of thecreated instance to the system state information collecting unit 27.

Next, a time point at which the processing contained in the identifiedrule is activated is recorded in the execution history storing unit 242in association with the identification information of the createdinstance (S102), and the created instance executes the associatedprocessing (S103). For example, when the identified rule is one that has“A” as the RULE_ID 2400 in FIG. 3, the instance executes processing of“scaling-in the Web tier”.

While the instance is executing the processing, an execution historyabout the processing is recorded in the execution history storing unit242 (S104). After the instance finishes the processing, the ruleprocessing unit 241 records, in the execution history storing unit 242,a result of executing the processing and the end time of the processingin association with the identification information of the instance(S105). The rule processing unit 241 then sends an execution completionnotification containing identification information of the instance thathas finished executing the processing, to the rule execution historyobtaining unit 25 and the system state information collecting unit 27(S106).

Through the processing of Steps S100 to S106, the rule executing unit 24completes a series of processing related to activation of a rule,execution of processing contained in the rule, and completion of theexecution of the processing.

Described next with reference to FIG. 5 is system state informationcollecting processing which is performed by the configurationinformation management unit 21 and the system state informationcollecting unit 27.

The system state information collecting unit 27 receives an activationnotification or an execution completion notification from the ruleprocessing unit 241 (S300), and sends an obtaining request to theconfiguration information management unit 21 as a request for the systemconfiguration information of the business operation system (S301).Receiving the system configuration information obtaining request (S400),the configuration information management unit 21 obtains the requestedsystem configuration information from the configuration informationstoring unit 22 (S401), and sends the obtained system configurationinformation to the system state information collecting unit 27 (S402).

The system configuration information sent to the system stateinformation collecting unit 27 in Step S402 has, for example, a dataconfiguration shown in FIG. 6. As shown in FIG. 6, the systemconfiguration information contains, in association with a RESOURCE_ID500, which is an identifier assigned to each resource, a TYPE 501, whichindicates the type of the resource, a STATE 502, which indicates theoperating state of the resource, a MACHINE_TYPE 503, which indicates thespecification of the resource, and the like.

A type indicating a server role such as Web server, AP server, or DBserver is stored as the TYPE 501. Information such as “RUNNING”, whichindicates that the resource is in operation, or “STANDBY”, whichindicates that the resource is an auxiliary resource or is on standby,is stored as the STATE 502. Stored as the MACHINE_TYPE 503 is a tag thatindicates a classification by the machine specification of the computerresource such as the CPU type, the clock frequency of the CPU, the clockfrequency of the front side bus (FSB) the memory size, the disk size,and the like.

Receiving the system configuration information from the configurationinformation management unit 21 (S302), the system state informationcollecting unit 27 refers to the received system configurationinformation to choose, as a resource for which information about theresource utilization state and information about the service utilizationstate are obtained, one among resources that are in operation and thathave not been chosen (S303).

The system state information collecting unit 27 obtains a measured valueof a metric related to the utilization state of the chosen resource(S304). The configuration of the data obtained by the system stateinformation collecting unit 27 in Step S304 is, for example, as shown inFIG. 7. The resource utilization state information that is obtained bythe system state information collecting unit 27 contains, for eachRESOURCE_ID 510, which is an identifier assigned to each resource,CPU-UTIL 511, which indicates the CPU utilization rate of the resourcein question, MEMORY_UTIL 512, which indicates the memory utilizationrate of this resource, and the like.

The resource utilization state information obtained by the system stateinformation collecting unit 27 may contain, in addition to the itemsshown in FIG. 7, the disk utilization rate, the network utilizationrate, a resource shutdown time, the amount of power consumption, theamount of heat generated, and the like.

The system state information collecting unit 27 next obtains a measuredvalue of a metric related to the service utilization state of the chosenresource (S305). The configuration of the data obtained by the systemstate information collecting unit 27 in Step S305 is, for example, asshown in FIG. 8. The information about the service utilization statethat is obtained by the system state information collecting unit 27contains, for each SERVICE_ID 520, which is an identifier assigned toeach service, REQUESTS 521, which indicate the count of requests for theservice in question, a RESPONSE_TIME 522, which indicates an averageresponse time of the service, FAILS 523, which indicate how manyrequests for the service have failed to meet a service level agreement(SLA), THROUGHPUT 524, which indicates how many requests have beenprocessed, and the like.

The service utilization state information obtained by the system stateinformation collecting unit 27 may contain, in addition to the itemsshown in FIG. 8, a connection error count, a sales figure oftransactions of a committed service, a service suspension time, and thelike.

Next, the system state information collecting unit 27 refers to thesystem configuration information to judge whether or not every resourcethat is in operation has been chosen once (S306). In cases where thereare resources that are in operation and have not been chosen (S306: No),the system state information collecting unit 27 repeats the processingof Step S303.

In cases where every resource that is in operation has been chosen once(S306: Yes), the system state information collecting unit 27 processesthe information about the resource utilization state into a format asshown in FIG. 9 whereas the information about the service utilizationstate is processed into a format as shown in FIG. 10, and stores theprocessed information in the rule operation history storing unit 26(S307). The system state information collecting unit 27 then ends thesystem state information collecting processing.

As shown in FIG. 9, the information about the resource utilization statecontains, for each INSTANCE_ID 2600, a PERIOD 2601, a TYPE 2602, aMACHINE_TYPE 2603, a STATE 2604, NUMBER_OF 2605, CPU_UTIL 2606, and thelike. Stored as the PERIOD 2601 is “START”, which indicates that thecollection of the system state information has been started due to anactivation notification, or “END”, which indicates that the collectionof the system state information has been started due to an executioncompletion notification. The NUMBER_OF 2605 indicates the count ofresources that have the same attributes in regard to the items from theTYPE 2602 to the STATE 2604. The CPU_UTIL 2606 indicates the average CPUutilization rate of resources that have the same attributes in regard tothe items from the TYPE 2602 to the STATE 2604.

The system state information collecting processing shown in FIG. 5 isexecuted each time an activation notification or an execution completionnotification is received from the rule processing unit 241. Accordingly,every piece of resource utilization state information that is collectedin one round of system state information collecting processing has thesame value as the INSTANCE_ID 2600 and has the same value as the PERIOD2601.

As shown in FIG. 10, the information about the service utilization statecontains, for each INSTANCE_ID 2610, a PERIOD 2611, a SERVICE_ID 2612,REQUESTS 2613, a RESPONSE_TIME 2614, FAIL S2615, THROUGHPUT 2616, andthe like. Similarly to the information about the resource utilizationstate, every piece of service utilization state information that iscollected in one round of system state information collecting processinghas the same value as the INSTANCE_ID 2610 and has the same value as thePERIOD 2611.

Returning to FIG. 4, the description is continued.

The execution history acquisition executing unit 250 of the ruleexecution history obtaining unit 25 receives an execution completionnotification from the rule processing unit 241 (S200), and increases theexecution completion notification reception count by 1 (S201). Theexecution history acquisition executing unit 250 compares theaccumulated reception count against an execution count entered inadvance through the specified information receiving unit 251 by the userof the performance evaluating apparatus for a rule 2 (S202). When theaccumulated reception count is equal to or larger than the executioncount (S202: Yes), the execution history acquisition executing unit 250executes processing of Step S204.

When the accumulated reception count is smaller than the execution count(S202: No), the execution history acquisition executing unit 250 judgeswhether or not a time having elapsed since a time point at which anexecution history has been received from the execution historytransmitting unit 243 last time is equal to or longer than an obtaininginterval entered in advance through the specified information receivingunit 251 (S203). In cases where the elapsed time is less than theobtaining interval (S203: No), the performance evaluating apparatus fora rule 2 ends the processing shown in the flow chart of FIG. 4.

In cases where the elapsed time is equal to or longer than the obtaininginterval (S203: Yes), the execution history acquisition executing unit250 inquires of the execution history transmitting unit 243 about theoperation state of the rule processing unit 241 (S204). The executionhistory transmitting unit 243 sends the operation state of the ruleprocessing unit 241 to the execution history acquisition executing unit250 in response to the inquiry (S107). The execution history acquisitionexecuting unit 250 then judges whether or not the rule processing unit241 is performing processing (S205). In cases where the rule processingunit 241 is performing processing (S205: Yes), the performanceevaluating apparatus for a rule 2 ends the processing shown in this flowchart.

In cases where the rule processing unit 241 is not performing processing(S205: No), the execution history acquisition executing unit 250searches the rule operation history storing unit 26 for instanceidentification information that is contained in the resource utilizationstate information described with reference to FIG. 9, or the serviceutilization information described with reference to FIG. 10, and forwhich no execution history has been stored in the rule operation historystoring unit 26. The execution history acquisition executing unit 250sends an execution history obtaining request that contains the retrievedinstance identification information, to the execution historytransmitting unit 243 (S206).

Receiving the execution history obtaining request, the execution historytransmitting unit 243 extracts, from the execution history storing unit242, an execution history that is associated with the instanceidentification information contained in the received execution historyobtaining request (S108). The execution history transmitting unit 243sends the extracted execution history to the execution historyacquisition executing unit 250 (S109).

The execution history acquisition executing unit 250 receives theexecution history from the execution history transmitting unit 243 andsaves the time of reception (S207). The execution history acquisitionexecuting unit 250 then stores the received execution history in therule operation history storing unit 26 (S208). The execution historystored in Step S208 by the execution history acquisition executing unit250 in the rule operation history storing unit 26 has, for example, aconfiguration shown in FIG. 11.

As shown in FIG. 11, the execution history contains, for each RULE_ID2620, an INSTANCE_ID 2621, a START_TIME 2622, which indicates the timeof activation of processing that is contained in a rule identified bythe RULE_ID 2620, an END_TIME 2623, which indicates the time ofcompletion of the execution of the processing that is contained in therule identified by the RULE_ID 2620, an EXECUTED_ACTION_ID 2624, whichindicates the processing that is contained in the rule identified by theRULE_ID 2620, a RESULT 2625, which indicates a result of executing theprocessing, and the like.

Through the steps shown in FIGS. 4 and 5, in cases of executing theprocessing that is associated with an event sent from a businessoperation system, system state information prior to the execution of theprocessing that is associated with an event sent from a businessoperation system and system state information after the execution ofthis processing are collected, an execution history of this processingis collected, and the collected system state information and executionhistory are stored in the rule operation history storing unit 26 inassociation with instance identification information.

The amount of data to be collected can be reduced by thus collectingdata only immediately before and after execution of processing that isassociated with an event instead of constantly collecting informationthat indicates the state of a monitored system. In addition, collectingsystem state information immediately before and after execution of theprocessing enables the evaluating apparatus to evaluate effects of therespective rules based on the collected information.

Described next with reference to FIG. 12 is processing that is performedby the performance evaluating apparatus for a rule 2 to output a ruleoperation history upon request from a user. The premise of FIG. 12 isthat the processing described with reference to FIGS. 4 and 5 has beenexecuted and that the rule operation history storing unit 26 hasaccumulated data.

First, the effect calculating unit 28 receives a rule operation historyinformation display request from the user of the performance evaluatingapparatus for a rule 2 through the input/output unit 20 (S500). In StepS500, the effect calculating unit 28 receives from the user a targetperiod, an item of system state information to be referred to, an outputformat, and the like.

Next, the effect calculating unit 28 analyzes the received displayrequest to extract necessary data from the rule operation historystoring unit 26 (S501). The effect calculating unit 28 obtains acalculation formula used in processing the data into a format that isrequested by the user, and processes the extracted data with the use ofthe obtained calculation formula (S502). The effect calculating unit 28stores, in advance, as shown in FIG. 13, for example, a DEFINITION 531,which indicates a calculation formula defined by the user, inassociation with a DEF_ID 530, which is an identifier assigned to eachcalculation formula.

With a calculation formula that has “E1” as the DEF_ID 530, the resourceutilization state information shown in FIG. 9 is searched for pieces ofdata that share the same INSTANCE_ID 2600, TYPE 2602, MACHINE_TYPE 2603,and STATE 2604. From these pieces of data, one that has “END” as thePERIOD 2601 and one that has “START” as the PERIOD 2601 are picked out,and the NUMBER_OF 2605 of the latter data is subtracted from theNUMBER_OF 2605 of the former data. The thus obtained value is multipliedby a FINANCIAL_COST 541 of a corresponding MACHINE_TYPE 540 of FIG. 14.The cost of increasing or reducing resources can be calculated in thismanner for each executed processing of a rule.

With a calculation formula that has “E2” as the DEF_ID 530, the resourceutilization state information shown in FIG. 9 is searched for pieces ofdata that share the same INSTANCE_ID 2600 and that have “END” as thePERIOD 2601, as well as pieces of data that share the same INSTANCE_ID2600 and that have “START” as the PERIOD 2601. The arithmetic average ofthe CPU_UTIL 2606 of the latter pieces of data is subtracted from thearithmetic average of the CPU_UTIL 2606 of the former pieces of data.The effective resource utilization degree can thus be calculated foreach executed processing of a rule.

With a calculation formula that has “E3” as the DEF_ID 530, the resourceutilization state information shown in FIG. 9 and information stored inadvance in the effect calculating unit 28 such as the information shownin FIG. 14 which indicate the resource cost and the amount of powerconsumption, are searched for pieces of data that share the sameINSTANCE_ID 2600, TYPE 2602, MACHINE_TYPE 2603, and STATE 2604. Fromthese pieces of data, one that has “END” as the PERIOD 2601 and one thathas “START” as the PERIOD 2601 are picked out to find resources forwhich the CPU_UTIL 2606 of the former data and the CPU_UTIL 2606 of thelatter data are both in a FULL_POWER state (a state in which the averageCPU utilization rate is 50% or higher in FIG. 14). An increase ordecrease in number of such resources is then calculated.

In the same manner, with a calculation formula that has “E3” as theDEF_ID 530, the resource utilization state information shown in FIG. 9and information stored in advance in the effect calculating unit 28,such as the information shown in FIG. 14, which indicate the resourcecost and the amount of power consumption, are searched for pieces ofdata that share the same INSTANCE_ID 2600, TYPE 2602, MACHINE_TYPE 2603,and STATE 2604. From these pieces of data, one that has “START” as thePERIOD 2601 and one that has “END” as the PERIOD 2601 are picked out tofind resources for which the CPU_UTIL 2606 of the former data and theCPU_UTIL 2606 of the latter data are both in a SAVED_POWER state (astate in which the average CPU utilization rate is lower than 50% inFIG. 14). An increase or decrease in number of such resources is thencalculated.

The calculated increase/decrease in number of resources that are in aFULL_POWER state is multiplied by a power consumption amount of theFULL_POWER state (300 W in FIG. 14 in cases where the MACHINE_TYPE 540is “SERVER_(—)00”). The calculated increase/decrease in number ofresources that are in a SAVED_POWER state is multiplied by a powerconsumption amount of the SAVED_POWER state (200 W in FIG. 14 in caseswhere the MACHINE_TYPE 540 is “SERVER_(—)00”). The products are addedand an increase or decrease in amount of power consumption can thus becalculated for each executed processing of a rule.

With a calculation formula that has “E4” as the DEF_ID 530, the serviceutilization state information shown in FIG. 10 and information stored inadvance in the effect calculating unit 28 such as the information shownin FIG. 15, which is a digitized business value of a service, aresearched for pieces of data that share the same INSTANCE_ID 2610, PERIOD2611, and SERVICE_ID 2612. The values of the THROUGHPUT 2616 of thesepieces of data are multiplied by the values of a SERVICE_VALUE 551, andthe products are summed up for each rule. For instance, in cases wherethe SERVICE_ID 2612 is “CATALOG”, the SERVICE_VALUE 551 is “100/tran” inFIG. 15. A sales contribution degree can thus be obtained for eachexecuted processing of a rule.

With a calculation formula that has “E4” as the DEF_ID 530, the serviceutilization state information shown in FIG. 10 and information stored inadvance in the effect calculating unit 28 such as the information shownin FIG. 15, which is a digitized business value of a service, aresearched for pieces of data that share the same INSTANCE_ID 2610, PERIOD2611, and SERVICE_ID 2612. The values of the FAILS 2615 of these piecesof data are multiplied by the values of a FAILS_LOST 552, and theproducts are summed up for each rule. For instance, in cases where theSERVICE_ID 2612 is “CATALOG”, the FAILS_LOST 552 is “0.2” in FIG. 15. Alost chance degree can thus be obtained for each executed processing ofa rule.

In the above-mentioned data processing, the effect calculating unit 28processes data with the use of a calculation formula that is stored inadvance in the effect calculating unit 28. Alternatively, the effectcalculating unit 28 may use a calculation formula that is entered by auser through the input/output unit 20 in processing data.

The effect calculating unit 28 outputs the processed data as, forexample, screen data that has a configuration shown in FIG. 16 (S503),and then ends the processing shown in this flowchart. The effectcalculating unit 28 may output data on a rule basis, or per executedprocessing of a rule. Also, the effect calculating unit 28 may calculatethe business value of a rule by a preset calculation formula from anexecution history of processing prescribed by the rule and from systemstate information on a processing basis of a service provided by acomputer system in which the processing prescribed by the rule isexecuted.

In the example of FIG. 16, the effect calculating unit 28 executesstatistical processing about processed data per processing for eachrule. As a result, an activation count 561, a success count 562, whichindicates how many times an objective has been accomplished, an averageexecution time 563, a resource utilization state 564, a serviceutilization state 565, and a business value 566 within a specifiedperiod are displayed for each RULE_ID 560 by the effect calculating unit28.

The resource utilization state 564 contains a resource increase/decreasein number, an average CPU utilization rate, a power consumptionincrease/decrease in amount, and the like. The service utilization state565 contains an average processing rate, a processing amount, and a rateof accomplishing a service level objective (SLO) such as “response time:within five seconds”. The business value 566 contains numerical valuesindicating business values that are calculated by the effect calculatingunit 28, for example, the resource cost, the effective resourceutilization degree, the sales contribution degree, the lost chancedegree, and the like. By referring to information as shown in FIG. 16, auser can evaluate each rule for its business values including the salesfigure, the lost chance amount, the cost reduction amount, the effectiveresource utilization degree, the environment burdening degree, and thelike.

FIG. 17 is a diagram showing an example of a hardware configuration forimplementing functions of the performance evaluating apparatus for arule 2. The performance evaluating apparatus for a rule 2 in thisembodiment has a central processing unit (CPU) 40, a memory 41 such as arandom access memory (RAM) or a read-only memory (ROM), an externalstorage 42 such as a hard disk drive (HDD), a communication interface(IF) 43, an input interface (IF) 44, an output interface (IF) 45, and amedia interface (IF) 46.

The CPU 40 operates in accordance with a program stored in the memory 41or in the external storage 42 to control the components of theperformance evaluating apparatus for a rule 2. The memory 41 stores aboot program, which is executed by the CPU 40 in activating theperformance evaluating apparatus for a rule 2, an application program, aprogram dependent on hardware of the performance evaluating apparatusfor a rule 2, data used by the CPU 40, and the like.

The communication IF 43 provides the CPU 40 with data received via themanagement network 15 from the load balancers 13, the computer resources16, or others. The communication IF 43 also sends data created by theCPU 40 to the load balancers 13, the computer resources 16, or othersvia the management network 15.

The CPU 40 controls an input device such as a keyboard or a mousethrough the input IF 44, and obtains data from the input device throughthe input IF 44. The CPU 40 controls an output device such as a liquidcrystal display (LCD) through the output IF 45, and outputs created datato the output device through the output IF 45.

The media IF 46 reads a program or data stored in a recording medium andprovides the read program or data to the CPU 40, the memory 41, and thecommunication IF 43. A program provided to the CPU 40 through the memory41 is stored in this recording medium. The program is read out of therecording medium by the media IF 46, is installed in the performanceevaluating apparatus for a rule 2 through the memory 41, and is executedby the CPU 40.

The external storage 42 stores data that is stored in the configurationinformation storing unit 22, the rule storing unit 240, the executionhistory storing unit 242, and the rule operation history storing unit26. A program installed and executed in the performance evaluatingapparatus for a rule 2 causes the performance evaluating apparatus for arule 2 to function as the input/output unit 20, the configurationinformation management unit 21, the configuration information storingunit 22, the event receiving unit 23, the rule executing unit 24, therule execution history obtaining unit 25, the rule operation historystoring unit 26, the system state information collecting unit 27, theeffect calculating unit 28, and the network IF unit 29.

The performance evaluating apparatus for a rule 2 reads these programsout of a recording medium through the media IF 46 for execution.Alternatively, the performance evaluating apparatus for a rule 2 mayobtain these programs from another device via a communication network. Arecording medium read by the media IF 46 is, for example, an opticalrecording medium such as a DVD or a PD, a magneto-optical recordingmedium such as an MO, a tape medium, a magnetic recording medium, or asemiconductor memory.

An embodiment of the present invention has thus been described.

As is clear from the above-mentioned description, the performanceevaluating apparatus for a rule 2 according to the present invention canefficiently collect information for evaluating from the businessviewpoint a rule that is used in the operation and management of aninformation system through autonomous control, by focusing on andcollecting the information immediately before and after the rule isexecuted.

The present invention is not limited to the embodiment described above,but can be modified in various ways within the gist of the invention.

For instance, while the rule execution history obtaining unit 25 in theflow chart of FIG. 4 does not make an attempt to obtain an executionhistory in the execution history storing unit 242 until the ruleprocessing unit 241 finishes executing processing that is contained in arule, the rule execution history obtaining unit 242 may attempt toobtain an execution history in the execution history storing unit 242regularly or when a preset condition is met, irrespective of whether therule processing unit 241 has finished executing the processing as shownin FIG. 18. Alternatively, the processing shown in the flow chart ofFIG. 18 may be executed in addition to the processing shown in the flowchart of FIG. 4.

The performance evaluating apparatus for a rule 2 in the aboveembodiment has the configuration information storing unit 22, the rulestoring unit 240, the execution history storing unit 242, and the ruleoperation history storing unit 26. In another embodiment, data stored inthese storing units may be stored in an external storage so that theperformance evaluating apparatus for a rule 2 obtains data in theexternal storage or write data in the external storage by communicatingwith the external storage over a network.

In the above embodiment, the performance evaluating apparatus for a rule2 which is a single machine implements the input/output unit 20, theconfiguration information management unit 21, the configurationinformation storing unit 22, the event receiving unit 23, the ruleexecuting unit 24, the rule execution history obtaining unit 25, therule operation history storing unit 26, the system state informationcollecting unit 27, the effect calculating unit 28, and the network IFunit 29. In another embodiment, each of these functions may beimplemented by one of multiple computers, and the multiple computerscommunicate with one another, whereby all of the multiple computersfunction as the performance evaluating apparatus for a rule 2.

1. A performance evaluating apparatus for an information system composedof multiple computers, the apparatus comprising: a rule operationhistory storing unit which stores an execution history in associationwith system state information before and after processing prescribed bya rule is executed in the information system, the execution historybeing a log of execution of the processing prescribed by the rule in theinformation system, the system state information including configurationinformation about a configuration of the information system, operationinformation that indicates an operating state of computer resourcesconstituting the information system, and utilization information thatindicates a utilization state of a service provided by the informationsystem; and an effect calculating unit which uses a preset calculationformula to calculate a business value of the rule from the executionhistory and the system state information.
 2. A performance evaluatingapparatus according to claim 1, wherein the effect calculating unit hasa calculation formula input receiving unit which receives thecalculation formula as input from outside.
 3. A performance evaluatingapparatus according to claim 1, wherein the calculation formula is forcalculating business value including at least one of the following items(1) through (5): (1) a resource increase/decrease cost obtained byclassifying computer resources that constitute a monitored informationsystem by machine type, counting the number of the computer resourcesfor each machine type before and after execution of processingprescribed by a rule, multiplying a difference between the pre-executionresource count and the post-execution resource count by a financial costfor each given machine type, and summing up the product for each machinetype of the computer resources that constitute the monitored informationsystem; (2) an effective resource utilization degree obtained bychecking which ones of the computer resources that constitute themonitored information system are in operation, obtaining an averageresource utilization rate of the computer resources that are inoperation before and after execution of the processing prescribed by therule respectively, the resource utilization rate including at least oneof a CPU utilization rate, a memory utilization rate, a disk utilizationrate, and a network utilization rate, and calculating a differencebetween average resource utilization rate of the pre-execution resourceutilization rate and average resource utilization rate of thepost-execution resource utilization rate; (3) an amount of powerconsumption obtained by classifying computer resources that constitutethe monitored information system by the power consumption amount,counting the number of computer resources that are classified as havingthe same power consumption amount before and after execution of theprocessing prescribed by the rule respectively, multiplying a differencebetween the pre-execution resource count and the post-execution resourcecount by the power consumption amount used in the classification withrespect to each classification, and summing up the product for thecomputer resources that constitute the information system; (4) a salescontribution degree obtained by counting how many requests are processedin a service that is provided by the monitored information system beforeand after execution of the processing prescribed by the rulerespectively, multiplying a difference between the pre-execution requestcount and the post-execution request count by a service value for eachprocessing that is defined for each service with respect to eachservice, and summing up the product for every service provided by themonitored information system; and (5) a lost chance degree obtained bycounting how many requests for a service that is provided by themonitored information system end up as an error before and afterexecution of the processing prescribed by the rule respectively,multiplying a difference between the pre-execution error count and thepost-execution error count by a weighting coefficient for one of anincrease and a decrease in number of lost chance which is specific toeach error count defined for each service with respect to each service,and summing up the product for every service that is provided by themonitored information system.
 4. A performance evaluating apparatusaccording to claim 1, wherein the effect calculating unit uses a presetcalculation formula to calculate from the execution history and thesystem state information, a business value of the rule per processingunit of a service that is provided by a computer system in which theprocessing prescribed by the rule has been executed.
 5. A performanceevaluating apparatus according to claim 1, further comprising: a ruleexecuting unit which executes processing prescribed in an associatedrule in accordance with the state of the monitored system and stores, asa rule execution history, a history about execution of processingprescribed by a rule; a system state information collecting unit whichcollects the system state information before and after execution of theprocessing prescribed by the rule respectively, and which stores thecollected system state information in the rule operation history storingunit; and a rule execution history obtaining unit which obtains, under agiven condition, the rule execution history stored in the rule executingunit.
 6. A performance evaluating apparatus according to claim 5,wherein the rule executing unit sends, before executing the processingprescribed by a rule, an execution start notification includingidentification information of a processing prescribed by the rule, tothe system state information collecting unit and, after the execution ofthe processing prescribed by a rule is finished, sends an executioncompletion notification including identification information of aprocessing prescribed by the rule, to the system state informationcollecting unit, wherein the system state information collecting unitcollects the system state information after receiving one of theexecution start notification and the execution completion notificationsent from the rule executing unit, and stores the collected system stateinformation in the rule operation history storing unit, in associationwith identification information of the processing prescribed by the ruleand a start/end symbol which indicates whether the system stateinformation has been collected at one of the start of the execution ofthe processing prescribed by the rule and of completion of the executionof the processing prescribed by the rule, and wherein the rule executionhistory obtaining unit comprises a specified information receiving unitwhich receives as input a condition for obtaining a rule executionhistory that is stored in the rule executing unit.
 7. A performanceevaluating apparatus according to claim 5, wherein processing in theexecution history obtaining unit is processed independently ofprocessing in the rule executing unit and processing in the system stateinformation collecting unit.
 8. A performance evaluating method for aperformance evaluating apparatus for an information system composed ofmultiple computers, the performance evaluating apparatus including arule operation history storing unit and an effect calculating unit, themethod comprising the steps of: recording, by the rule operation historystoring unit, an execution history of a rule in association with systemstate information before and after execution of processing prescribed bythe rule, the system state information including configurationinformation about a configuration of a monitored information system,operation information that indicates an operating state of computerresources constituting the monitored information system, and utilizationinformation that indicates a utilization state of a service provided bythe monitored information system; and calculating, by the effectcalculating unit, using a preset calculation formula, a business valueof the rule from the execution history and from the system stateinformation.
 9. A performance evaluating method according to claim 8,wherein the effect calculating unit executes the step of receiving thecalculation formula as an input from outside.
 10. A performanceevaluating method according to claim 8, wherein the calculation formulais for calculating business value including at least one of thefollowing items (1) through (5): (1) a resource increase/decrease costobtained by classifying computer resources that constitute the monitoredinformation system by machine type, counting the number of the computerresources for each machine type before and after execution of processingprescribed by a rule, multiplying a difference between the pre-executionresource count and the post-execution resource count by a financial costfor each given machine type, and summing up the product for each machinetype of the computer resources that constitute the monitored informationsystem; (2) an effective resource utilization degree obtained bychecking which ones of the computer resources that constitute themonitored information system are in operation, obtaining an averageresource utilization rate of the computer resources that are inoperation before and after execution of the processing prescribed by therule respectively, the resource utilization rate including at least oneof a CPU utilization rate, a memory utilization rate, a disk utilizationrate, and a network utilization rate, and calculating a differencebetween average resource utilization rate of the pre-execution resourceutilization rate and average resource utilization rate of thepost-execution resource utilization rate; (3) an amount of powerconsumption obtained by classifying computer resources that constitutethe monitored information system by the power consumption amount,counting the number of computer resources that are classified as havingthe same power consumption amount before and after execution of theprocessing prescribed by the rule respectively, multiplying a differencebetween the pre-execution resource count and the post-execution resourcecount by the power consumption amount used in the classification withrespect to each classification, and summing up the product for thecomputer resources that constitute the information system; (4) a salescontribution degree obtained by counting how many requests are processedin a service that is provided by the monitored information system beforeand after execution of the processing prescribed by the rulerespectively, multiplying a difference between the pre-execution requestcount and the post-execution request count by a service value for eachprocessing that is defined for each service with respect to eachservice, and summing up the product for every service provided by themonitored information system; and (5) a lost chance degree obtained bycounting how many requests for a service that is provided by themonitored information system end up as an error before and afterexecution of the processing prescribed by the rule respectively,multiplying a difference between the pre-execution error count and thepost-execution error count by a weighting coefficient for one of anincrease and a decrease in number of lost chance which is specific toeach error count defined for each service with respect to each service,and summing up the product for every service that is provided by themonitored information system.
 11. A performance evaluating methodaccording to claim 8, wherein the effect calculating unit uses a presetcalculation formula to calculate from the execution history and thesystem state information, a business value of the rule per processingunit of a service that is provided by a computer system in which theprocessing prescribed by the rule has been executed.
 12. A performanceevaluating method according to claim 8, wherein the performanceevaluating apparatus further includes a rule executing unit, a systemstate information collecting unit, and a rule execution historyobtaining unit, wherein the rule executing unit executes a step ofexecuting processing prescribed in an associated rule in accordance withthe state of the monitored system and storing, as a rule executionhistory, a history about execution of processing prescribed by a rule,wherein the system state information collecting unit executes a step ofcollecting the system state information before and after execution ofthe processing prescribed by the rule respectively, and a step ofstoring the collected system state information in the rule operationhistory storing unit, and wherein the rule execution history obtainingunit executes a step of obtaining, under a given condition, the ruleexecution history stored in the rule executing unit.
 13. A performanceevaluating method according to claim 12, wherein the rule executing unitfurther executes the steps of: sending, before executing the processingprescribed by a rule, an execution start notification includingidentification information of a processing prescribed by the rule, tothe system state information collecting unit; and sending, after theexecution of the processing prescribed by a rule is finished, anexecution completion notification including identification informationof a processing prescribed by the rule, to the system state informationcollecting unit, wherein the system state information collecting unitfurther executes the steps of: receiving one of the execution startnotification and the execution completion notification sent from therule executing unit; collecting the system state information afterreceiving the notification; and storing the collected system stateinformation in the rule operation history storing unit, in associationwith identification information of the processing prescribed by the ruleand a start/end symbol which indicates whether the system stateinformation has been collected at one of the start of the execution ofthe processing prescribed by the rule and of completion of the executionof the processing prescribed by the rule, and wherein the rule executionhistory obtaining unit further executes a step of receiving as input acondition for obtaining a rule execution history that is stored in therule executing unit.
 14. A performance evaluating method according toclaim 12, wherein the rule execution history obtaining step by the ruleexecution history obtaining unit is executed independently execution ofthe steps by the rule executing unit and execution of the steps by thesystem state information collecting unit.
 15. A program that causes aperformance evaluating system for an information system composed ofmultiple computers to comprise a rule operation history storing functionand an effect calculating function, wherein the rule operation historystoring function stores an execution history of a rule in associationwith system state information before and after execution of processingprescribed by the rule is executed in the information system, the systemstate information including configuration information about aconfiguration of a monitored information system, operation informationthat indicates an operating state of computer resources constituting themonitored information system, and utilization information that indicatesa utilization state of a service provided by the monitored informationsystem, and wherein the effect calculating function uses a presetcalculation formula to calculate a business value of the rule from theexecution history and the system state information.
 16. A programaccording to claim 15, wherein the effect calculating function has acalculation formula input receiving function which receives thecalculation formula as an input from outside.
 17. A program according toclaim 15, wherein the calculation formula is for calculating businessvalue including at least one of the following items (1) through (5): (1)a resource increase/decrease cost obtained by classifying computerresources that constitute the monitored information system by machinetype, counting the number of the computer resources for each machinetype before and after execution of processing prescribed by a rule,multiplying a difference between the pre-execution resource count andthe post-execution resource count by a financial cost for each givenmachine type, and summing up the product for each machine type of thecomputer resources that constitute the monitored information system; (2)an effective resource utilization degree obtained by checking which onesof the computer resources that constitute the monitored informationsystem are in operation, obtaining an average resource utilization rateof the computer resources that are in operation before and afterexecution of the processing prescribed by the rule respectively, theresource utilization rate including at least one of a CPU utilizationrate, a memory utilization rate, a disk utilization rate, and a networkutilization rate, and calculating a difference between average resourceutilization rate of the pre-execution resource utilization rate andaverage resource utilization rate of the post-execution resourceutilization rate; (3) an amount of power consumption obtained byclassifying computer resources that constitute the monitored informationsystem by the power consumption amount, counting the number of computerresources that are classified as having the same power consumptionamount before and after execution of the processing prescribed by therule respectively, multiplying a difference between the pre-executionresource count and the post-execution resource count by the powerconsumption amount used in the classification with respect to eachclassification, and summing up the product for the computer resourcesthat constitute the information system; (4) a sales contribution degreeobtained by counting how many requests are processed in a service thatis provided by the monitored information system before and afterexecution of the processing prescribed by the rule respectively,multiplying a difference between the pre-execution request count and thepost-execution request count by a service value for each processing thatis defined for each service with respect to each service, and summing upthe product for every service provided by the monitored informationsystem; and (5) a lost chance degree obtained by counting how manyrequests for a service that is provided by the monitored informationsystem end up as an error before and after execution of the processingprescribed by the rule respectively, multiplying a difference betweenthe pre-execution error count and the post-execution error count by aweighting coefficient for one of an increase and a decrease in number oflost chance which is specific to each error count defined for eachservice with respect to each service, and summing up the product forevery service that is provided by the monitored information system. 18.A program according to claim 15, wherein the effect calculating functionfurther has a function which uses a preset calculation formula tocalculate from the execution history and the system state information, abusiness value of the rule per processing unit of a service that isprovided by a computer system in which the processing prescribed by therule has been executed.
 19. A computer-readable program according toclaims 15, wherein the program causes the performance evaluatingapparatus to further function as a rule executing function, a systemstate information collecting function, and a rule execution historyobtaining function, wherein the rule executing function executesprocessing prescribed in an associated rule in accordance with the stateof the monitored system and stores, as a rule execution history, ahistory about execution of processing prescribed by a rule, wherein thesystem state information collecting function collects the system stateinformation before and after execution of the processing prescribed bythe rule respectively, and has the rule operation history storingfunction store the collected system state information, and wherein therule execution history obtaining function obtains, under a givencondition, the rule execution history stored by the rule executingfunction.
 20. A program according to claim 19, wherein the ruleexecuting function sends, before executing the processing prescribed bya rule, an execution start notification including identificationinformation of a processing prescribed by the rule to the system stateinformation collecting function and, after the execution of theprocessing prescribed by a rule is finished, sends an executioncompletion notification including identification information of aprocessing prescribed by the rule to the system state informationcollecting function, wherein the system state information collectingfunction collects the system state information, after receiving one ofthe execution start notification and the execution completionnotification sent from the rule executing function, and has the ruleoperation history storing function store the collected system stateinformation in association with identification information of theprocessing prescribed by the rule and a start/end symbol which indicateswhether the system state information has been collected at one of thestart of the execution of the processing prescribed by the rule and ofcompletion of the execution of the processing prescribed by the rule,and wherein the rule execution history obtaining function receives asinput a condition for obtaining a rule execution history that is storedby the rule executing function.
 21. A program according to claim 19,wherein processing that is performed by the rule execution historyobtaining function is executed independently of the rule executingfunction and the system state information collecting function.